As datasets continue to expand in size and complexity, these models have become increasingly sophisticated, with deeper architectures and greater expressive power. Despite these advances, DNNs trained on imbalanced class distributions often exhibit a tendency to favor majority classes, leading to degraded performance on underrepresented classes [18, 39, 27, 17]. Because many real-world datasets follow long-tailed distributions in which minority classes can contain critical and informative patterns, developing methods that enable DNNs to learn effectively from imbalanced data is essential to prevent the loss of valuable information from these rare classes [26, 34, 16]. Moreover, data encountered in real-world applications are frequently multi-modal, meaning that observations originate from heterogeneous sources [6, 29, 7, 35]. To make effective use of such heterogeneous inputs, a wide range of multi-modal learning approaches have been proposed that exploit complementary information across modalities to enhance predictive performance [10, 5]. Common strategies integrate multiple modalities into a unified representation, using techniques that span from straightforward feature-level concatenation [19, 11, 12] to more sophisticated neural architectures that learn joint representations in an end-to-end manner [20, 32]. Although prior research has extensively studied class imbalance and multi-modal data separately, relatively little attentionhas beengiven to settings where bothchallenges arise si2 multaneously. Developing methods that can effectively handle long-tailed class distributions in conjunction with multi-modal inputs is therefore essential in many real-world applications. In the medical domain, for instance, datasets often contain far more samples from healthy individuals than from patients with specific conditions, while also encompassing diverse datatypes such asimagingdata(e.g., X-rays)alongsideauxiliary informationincluding demographics and clinical histories.

VLMEvaluation To evaluate two VLMs (Frozen in Time [1] and VideoCLIP [13]), we use a hybrid approach that leverages both prototypical networks [11] and the video-language similarity metrics learned by both models. Below, we show an ablation study where we use only the video prototype networks. We show the performance of using only language similarity in the few-shot case to demonstrate the effects of sample removal, and we also show the effects of our hybrid weighting scheme, where we weight the language embeddings five times more than the video embeddings when constructing the hybrid prototype (as opposed to equal weighting during the regular hybrid approach). Although we perform our ablation study with Frozen-in-Time, and use the same weighting scheme and prototype strategy for VideoCLIP as well. For this study, we show activity and sub-activity classification accuracy in the 5-shot case. We visualize whether a given method uses language, video, or both to create its prototype embeddings.

Semi-supervised learning (SSL) offers a robust framework for harnessing the potential of unannotated data. Traditionally, SSL mandates that all classes possess labeled instances.

Continual semantic segmentation aims to learn new classes while maintaining the information from the previous classes.